在带有多个扬声器的视频中，主动扬声器检测（ASD）是一项具有挑战性的任务，因为它需要在长时间的暂时窗口上学习有效的视听功能和时空相关性。在本文中，我们提出了一种新颖的时空图形学习框架，可以解决复杂的任务，例如ASD。为此，视频框架中的每个人首先在该框架的唯一节点中编码。对应于跨帧的单个人的节点已连接以编码其时间动力学。帧中的节点也连接到编码人际关系。因此，咒语将ASD减少到节点分类任务。重要的是，咒语能够在所有节点上为所有节点上的长时间环境推理，而无需依赖计算昂贵的完全连接的图形神经网络。通过对Ava-Activespeaker数据集进行的广泛实验，我们证明了基于图形的表示形式可以显着改善主动扬声器检测性能，因为其明确的空间和时间结构。拼写优于所有先前的最新方法，同时需要大大降低内存和计算资源。我们的代码可在https://github.com/sra2/spell上公开获取

动态场景图表形式的结构化视频表示是有关多个视频理解任务的有效工具。与场景图的任务相比，由于场景的时间动态和预测的固有时间波动，动态场景图生成是更具挑战性。我们表明捕获长期依赖性是有效生成动态场景图的关键。我们通过从视频中构造一致的长期对象轨迹来介绍检测跟踪 - 识别范例，然后是捕获对象和视觉关系的动态。实验结果表明，我们的动态场景图检测变压器（DSG-DETR）在基准数据集动作基因组上的显着余量优于最先进的方法。我们还进行消融研究并验证所提出的方法的每个组成部分的有效性。

我们通过新的框架解决了主动扬声器检测问题，称为法术，从而了解远程多模式图来编码音频和视觉数据之间的模态关系。我们将主动扬声器检测作为了解长期依赖项的节点分类任务。我们首先从视频构造图形，以便每个节点对应一个人。表示在定义的时间窗口中它们之间相同身份的共享边缘的节点。同一视频帧中的节点也连接以编码人际交互。通过对AVA-ActiveSpeaker数据集的广泛实验，我们证明了基于学习的基于图形的表示，由于其明确的空间和时间结构，显着提高了整体性能。法术优于若干相关基线，并在现有技术的比例下执行，同时需要较小的计算成本阶数。

代理商通信可能会显着提高需要协调以实现共享目标的多代理任务的性能。事先工作表明，可以使用多智能体增强学习和消息传递网络架构学习代理商通信协议。然而，这些模型使用不受约束的广播通信模型，其中代理在每个步骤中与所有其他代理通信，即使任务不需要它。在现实世界应用中，如果通信可以受系统限制的限制，如带宽，电源和网络容量，则可能需要减少发送的消息的数量。在这项工作中，我们探讨了最大限度地减少通信的简单方法，同时在多任务学习中最大化性能：同时优化特定于任务的目标和通信惩罚。我们表明，目的可以使用强化和Gumbel-Softmax Reparameterization优化。我们介绍了两种稳定培训的技术：50％的培训和消息转发。在仅50％的剧集中培训沟通惩罚可防止我们的模型关闭外向消息。其次，重复消息先前接收的消息有助于模型保留信息，并进一步提高性能。通过这些技术，我们表明我们可以减少75％的通信，没有损失。

We demonstrate a Physics-informed Neural Network (PINN) based model for real-time health monitoring of a heat exchanger, that plays a critical role in improving energy efficiency of thermal power plants. A hypernetwork based approach is used to enable the domain-decomposed PINN learn the thermal behavior of the heat exchanger in response to dynamic boundary conditions, eliminating the need to re-train. As a result, we achieve orders of magnitude reduction in inference time in comparison to existing PINNs, while maintaining the accuracy on par with the physics-based simulations. This makes the approach very attractive for predictive maintenance of the heat exchanger in digital twin environments.

Deep Learning and Machine Learning based models have become extremely popular in text processing and information retrieval. However, the non-linear structures present inside the networks make these models largely inscrutable. A significant body of research has focused on increasing the transparency of these models. This article provides a broad overview of research on the explainability and interpretability of natural language processing and information retrieval methods. More specifically, we survey approaches that have been applied to explain word embeddings, sequence modeling, attention modules, transformers, BERT, and document ranking. The concluding section suggests some possible directions for future research on this topic.

Foveated imaging provides a better tradeoff between situational awareness (field of view) and resolution and is critical in long-wavelength infrared regimes because of the size, weight, power, and cost of thermal sensors. We demonstrate computational foveated imaging by exploiting the ability of a meta-optical frontend to discriminate between different polarization states and a computational backend to reconstruct the captured image/video. The frontend is a three-element optic: the first element which we call the "foveal" element is a metalens that focuses s-polarized light at a distance of $f_1$ without affecting the p-polarized light; the second element which we call the "perifoveal" element is another metalens that focuses p-polarized light at a distance of $f_2$ without affecting the s-polarized light. The third element is a freely rotating polarizer that dynamically changes the mixing ratios between the two polarization states. Both the foveal element (focal length = 150mm; diameter = 75mm), and the perifoveal element (focal length = 25mm; diameter = 25mm) were fabricated as polarization-sensitive, all-silicon, meta surfaces resulting in a large-aperture, 1:6 foveal expansion, thermal imaging capability. A computational backend then utilizes a deep image prior to separate the resultant multiplexed image or video into a foveated image consisting of a high-resolution center and a lower-resolution large field of view context. We build a first-of-its-kind prototype system and demonstrate 12 frames per second real-time, thermal, foveated image, and video capture in the wild.

K-means++ is an important algorithm to choose initial cluster centers for the k-means clustering algorithm. In this work, we present a new algorithm that can solve the $k$-means++ problem with near optimal running time. Given $n$ data points in $\mathbb{R}^d$, the current state-of-the-art algorithm runs in $\widetilde{O}(k )$ iterations, and each iteration takes $\widetilde{O}(nd k)$ time. The overall running time is thus $\widetilde{O}(n d k^2)$. We propose a new algorithm \textsc{FastKmeans++} that only takes in $\widetilde{O}(nd + nk^2)$ time, in total.

Motivated by the goal of endowing robots with a means for focusing attention in order to operate reliably in complex, uncertain, and time-varying environments, we consider how a robot can (i) determine which portions of its environment to pay attention to at any given point in time, (ii) infer changes in context (e.g., task or environment dynamics), and (iii) switch its attention accordingly. In this work, we tackle these questions by modeling context switches in a time-varying Markov decision process (MDP) framework. We utilize the theory of bisimulation-based state abstractions in order to synthesize mechanisms for paying attention to context-relevant information. We then present an algorithm based on Bayesian inference for detecting changes in the robot's context (task or environment dynamics) as it operates online, and use this to trigger switches between different abstraction-based attention mechanisms. Our approach is demonstrated on two examples: (i) an illustrative discrete-state tracking problem, and (ii) a continuous-state tracking problem implemented on a quadrupedal hardware platform. These examples demonstrate the ability of our approach to detect context switches online and robustly ignore task-irrelevant distractors by paying attention to context-relevant information.

While large pretrained language models (PLMs) demonstrate incredible fluency and performance on many natural language tasks, recent work has shown that well-performing PLMs are very sensitive to what prompts are feed into them. Even when prompts are semantically identical, language models may give very different answers. When considering safe and trustworthy deployments of PLMs we would like their outputs to be consistent under prompts that mean the same thing or convey the same intent. While some work has looked into how state-of-the-art PLMs address this need, they have been limited to only evaluating lexical equality of single- or multi-word answers and do not address consistency of generative text sequences. In order to understand consistency of PLMs under text generation settings, we develop a measure of semantic consistency that allows the comparison of open-ended text outputs. We implement several versions of this consistency metric to evaluate the performance of a number of PLMs on paraphrased versions of questions in the TruthfulQA dataset, we find that our proposed metrics are considerably more consistent than traditional metrics embodying lexical consistency, and also correlate with human evaluation of output consistency to a higher degree.